The face recognition technique is a technique of authenticating a person by extracting a plurality of feature points from a face image of the person, digitizing the features, and matching the digitized feature points with other image. The face recognition device is capable of recognizing a somewhat disguised face or a face in a photograph of some decades ago based on a plurality of extracted feature points. As such, face recognition devices have been widely utilized in a field such as personal identification.
However, the face of a person sometimes shows a different look in a face image or a portion of the face may be hidden due to factors, such as the angle, posture, brightness, position, and size of the face. It is difficult to extract feature points of the invisible portion of a face. For such reasons, the face recognition technique embodies a problem in which recognition of a face image cannot be accurately performed.
To address this problem, recently, a three-dimensional face recognition technique for achieving accurate recognition of a face in consideration of the invisible portion of the face has been proposed. This technique acquires stereoscopic information of a face using three-dimension sensors. Then, this technique can extract significant feature points of the contour of the eye orbit, nose, chin, or the like from the stereoscopic information, generate a face image of a different angle than the input face image, and compare the face image with other face image.
For example, the image recognition device disclosed in PLT 1 generates a three-dimensional face model by using feature point information of a face detected from an input image and three-dimensional shape information of a face that is registered in advance. Next, the image recognition device of PLT 1 calculates a geometric transformation parameter by using the generated three-dimensional face model. Based on this parameter, the image recognition device of PLT 1 generates a plurality of face pattern images (two-dimensional face images) of different postures and extracts facial feature vectors from the generated face pattern images. Then, the image recognition device of PLT 1 calculates a similarity between the facial feature vectors of registered images that is registered in advance and the feature vectors of the generated face pattern images, and recognizes the face image. By using the method as above, the image recognition device of PLT 1 can align (normalize) the position, size or the like of a face in a face image to reference data.
However, the method of PLT 1 sometimes cannot accurately detect the feature points of a face detected from an input image due to the angle, posture, or brightness of the face. In particular, due to a change of the posture of a face (for example, due to a hidden portion (self-occlusion) that happens on the side or contour of the nose), the method of PLT 1 possibly erroneously detects the feature point information of the accurate position of the nose.
FIG. 13 is a diagram illustrating an example of erroneously detecting likely positions as feature points in an image when the feature points that are supposed to exist are hidden due to occlusion, by taking an example of face contour points.
As illustrated in FIG. 13, the face contour points (black dots) detected in a frontal face image 100 and the face contour points detected in an oblique right face image 200 are located at different positions in the strict sense. As illustrated in the face image 300, the originally existing positions of the face contour points in the frontal face image 100, when the face is three-dimensionally rotated, are located inner side of the face than the face contour points detected in the oblique right face image 200. Thus, the face contour points detected in the oblique right face image 200 are not accurate feature points.
As such, a correct facial feature vector cannot be extracted from a face pattern image that is generated by calculating a geometric transformation parameter by using such information as detected from the oblique right face image 200. Thus, the technique of PLT 1 has the above-mentioned problem.
Two approaches can be considered to solve the technical problem described in PLT 1. The first approach is an approach of estimating a geometric transformation parameter without using feature points that could not be detected at accurate positions.
The first approach, for example, can be realized by using a robust estimation method represented by Random Sample Consensus (RANSAC). In the first approach, the visible portion of the face can be accurately normalized. However, the image of the area hidden in the deep side of the screen is destructed due to normalization.
Concretely, a phenomenon (loss), in which the area where the texture of a face is supposed to exist is mixed with the background texture, occurs. As such, the first approach has the above problem.
The second approach is an approach of correcting the feature points that could not be detected at accurate positions to the accurate positions.
The method of correcting feature points in the second approach includes, for example, a method disclosed in PLT 2. The method of PLT 2 generates variations of feature point positions in partial spaces (for example, three patterns of front, right, and left orientations) in advance, and obtains corrected feature point positions based on projection or back projection in the partial spaces.
Further, NPL 1 discloses a technique of maintaining, in advance, correction quantities when a standard stereoscopic face model is rotated for respective angles in a table and selecting an appropriate correction quantity from the table based on the angle of a face in a two-dimensional image.
It should be noted that PLT 3 describes a method that is used in the Description of Embodiments.